Unit testing and evaluating LLMs

How do you test a component whose defining feature is that it gives a different answer every time? You split the problem in two:

1. Unit tests for your logic: history handling, prompt building, tool wiring. These must be deterministic, so you replace the model with a stub.
2. Evaluation for the model’s output quality — you don’t assert exact strings, you score the response against a reference and assert the score clears a bar.

Microsoft.Extensions.AI supports both, and the abstraction is what makes the first one possible.

Substitute the model with a stub

Because your code depends on IChatClient, a test can supply a fake implementation with fixed output. From StubChatClient.cs:

public sealed class StubChatClient(Func<IEnumerable<ChatMessage>, string> respond) : IChatClient
{
    public int CallCount { get; private set; }
    public IReadOnlyList<ChatMessage>? LastMessages { get; private set; }

    public Task<ChatResponse> GetResponseAsync(
        IEnumerable<ChatMessage> messages, ChatOptions? options = null, CancellationToken ct = default)
    {
        CallCount++;
        LastMessages = messages.ToList();
        return Task.FromResult(new ChatResponse(new ChatMessage(ChatRole.Assistant, respond(LastMessages))));
    }

    // GetStreamingResponseAsync, GetService, Dispose ...
}

The stub records what it was called with, so tests can assert on the messages your code sent. That is where the real logic lives. In production you would reach for a mocking library, but writing it out shows there is no magic — it is just an interface.

Testing the conversation logic

The unit under test is a trimmed Conversation class (Conversation.cs). The tests (ConversationTests.cs) prove the bookkeeping without ever touching a model:

[Test]
public async Task SendAsync_SendsFullHistoryEachTurn()
{
    var stub = new StubChatClient(_ => "ok");
    var conversation = new Conversation(stub);

    await conversation.SendAsync("first");
    await conversation.SendAsync("second");

    // Second call must carry user/assistant/user = 3 messages.
    Assert.That(stub.CallCount, Is.EqualTo(2));
    Assert.That(stub.LastMessages, Has.Count.EqualTo(3));
    Assert.That(stub.LastMessages![^1].Text, Is.EqualTo("second"));
}

This directly guards the Your first IChatClient rule that stateless clients need the full history resent each turn. A regression here — the amnesia bug — fails the test in milliseconds, offline, every time.

The stub can even simulate memory by reading the history it is handed, which proves the production code preserves it:

var stub = new StubChatClient(messages =>
{
    var firstUserMessage = messages.First(m => m.Role == ChatRole.User).Text;
    return $"You first said: {firstUserMessage}";
});

await conversation.SendAsync("my favourite colour is blue");
var second = await conversation.SendAsync("what did I say?");

Assert.That(second, Does.Contain("blue"));

Evaluating output quality

For the model’s actual words you need evaluation. Microsoft.Extensions.AI splits evaluators into two families.

Lexical evaluators: offline, deterministic, CI-safe

Microsoft.Extensions.AI.Evaluation.NLP provides BLEU, GLEU and F1. They compare response text to reference text mathematically — no model, no network — so they belong in CI. From EvaluationTests.cs:

var evaluator = new BLEUEvaluator();

var response = new ChatResponse(new ChatMessage(
    ChatRole.Assistant, "The quick brown fox jumps over the lazy dog"));

var context = new BLEUEvaluatorContext(
    references: ["The quick brown fox jumps over the lazy dog"]);

var result = await evaluator.EvaluateAsync(
    messages: [new ChatMessage(ChatRole.User, "Describe the sentence.")],
    modelResponse: response,
    chatConfiguration: null,          // null: no judge model needed
    additionalContext: [context]);

var metric = result.Get<NumericMetric>(BLEUEvaluator.BLEUMetricName);
Assert.That(metric.Value, Is.GreaterThan(0.9));

An unrelated answer scores near zero:

// response: "Quarterly revenue rose twelve percent year over year"
// reference: "The quick brown fox jumps over the lazy dog"
Assert.That(metric.Value, Is.LessThan(0.3));

These evaluators are perfect for regression-testing a summariser or translator against a fixed golden set: you assert the score never drops below a threshold.

Model-graded evaluators: the “LLM as judge”

Microsoft.Extensions.AI.Evaluation.Quality provides Equivalence, Coherence, Relevance, Groundedness and more. These use a second model as the judge — you give it a reference meaning and it scores 1–5 whether the response matches. Sketch:

var equivalence = new EquivalenceEvaluator();
var ctx = new EquivalenceEvaluatorContext("The user's favourite colour is blue");

var result = await equivalence.EvaluateAsync(
    [new ChatMessage(ChatRole.User, "What is my favourite colour?")],
    modelResponse,
    new ChatConfiguration(judgeChatClient),   // a real IChatClient as judge
    [ctx]);

var metric = result.Get<NumericMetric>(EquivalenceEvaluator.EquivalenceMetricName);
Assert.That(metric.Value, Is.GreaterThanOrEqualTo(4));

Because a judge model is non-deterministic and needs a running LLM, keep these out of the fast unit run — mark them [Explicit]/[Category("Integration")] and run them on a schedule against a real endpoint, not on every commit.

Run it

dotnet test 04.Testing

Passed!  - Failed: 0, Passed: 6, Skipped: 0, Total: 6

All six run offline: four stub-based logic tests and two BLEU evaluations. No Ollama required.

The testing pyramid for AI code

• Base: logic tests with a stub IChatClient. Fast, deterministic, the bulk of your suite. They assert on what you send and how you handle what comes back.
• Middle: lexical evaluation (BLEU/GLEU/F1). Deterministic, offline, CI-safe regression guards on quality against golden references.
• Top: model-graded evaluation. Slow, non-deterministic, run occasionally against a real endpoint to catch drift.

The abstraction is what makes the base layer possible at all: swapping a real model for a fake is a one-line constructor change precisely because your code never named a concrete provider.